Skip to content

Conversation

lcian
Copy link
Member

@lcian lcian commented Aug 28, 2025

Description

This changes how the tracing integration works with regards to span names and operations.
(In the analysis below, I assume the spans are all created with the tracing::instrument macro)

Previous behavior

  • Sentry span name = <module>::<function>
    • unless a message field is set, in which case: Sentry span name = tracing span message
    • this message field is arbitrary, not documented, and doesn't align with our conventions
    • note that it's different from the tracing span name, this requires an attribute called message
  • Sentry op = function name
Screenshot 2025-08-28 at 16 16 01

Behavior after this PR

  • Sentry span name = tracing span name (= function name, when using the tracing::instrument macro)
  • Sentry op = default, as we have no way to infer an op that makes sense - it will be hidden in the UI
  • Users can set 3 additional fields that have special meaning for our tracing Subscriber and override values on the corresponding Sentry span:
    • sentry.name
    • sentry.op
    • sentry.trace: while not related to this issue, I noticed that there is confusion on how one would achieve distributed tracing. This makes it possible without having to use the Sentry tracing API and understand the interplay between Sentry and the tracing crate.
Screenshot 2025-08-28 at 16 15 51

Advantages of the change

  • The new conversion is more aligned with Sentry conventions, documented, and gives additional capabilities to users
  • It's also similar to the approach taken by tracing-opentelemetry, which offers a otel.name special property

Drawbacks of the change

  • We break existing customers that might be using span metrics/dashboards/queries based on the old span name - they will have to change them
  • If you have 2 functions with the same name in different modules, they will now have the same span name. We could mitigate this by writing the info from span.metadata.target into span attributes, like we do for logs to surface function and line

Closes

Close #880
Close #879

@lcian lcian changed the base branch from master to lcian/test/fix-doctests August 28, 2025 13:20
Copy link

codecov bot commented Aug 28, 2025

Codecov Report

❌ Patch coverage is 78.49462% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.43%. Comparing base (045c2e2) to head (5fdd55f).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #887      +/-   ##
==========================================
+ Coverage   73.39%   73.43%   +0.04%     
==========================================
  Files          64       64              
  Lines        7423     7502      +79     
==========================================
+ Hits         5448     5509      +61     
- Misses       1975     1993      +18     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Base automatically changed from lcian/test/fix-doctests to master August 28, 2025 13:22
lcian added 2 commits August 28, 2025 15:25
Corrected punctuation in the CHANGELOG regarding Sentry span overrides.
Copy link
Member

@Swatinem Swatinem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, this looks a lot cleaner indeed.

I think its fair to use just the function name, from the users perspective, I think that makes sense. Although I also might want to see the fully qualified name sometimes, in particular for method names.
its a bit unfortunate that tracing does not differentiate between an impl method and a free function here.
Ideally, it should qualify methods with the impl (such as Vec::extend_from_slice).
I think not qualifying anything could lead to confusion for example if you have str::from_utf8 vs String::from_utf8, which are two different methods.

But okay, I guess that is a minor annoyance.

@lcian
Copy link
Member Author

lcian commented Aug 29, 2025

Thanks @Swatinem, that's a fair point.
We could still report the fully qualified name as the op, so the information in the UI would be the same as before but swapped. Maybe that's the way to go, as default doesn't bring any value to the user anyway, might as well "abuse" the field to report the full name.

@lcian lcian marked this pull request as ready for review August 29, 2025 12:32
@Swatinem
Copy link
Member

One thing to try would be whether it actually is the "fully qualified" name, because just concatenating the module to the name, does it also include the type name for impl methods?

@lcian
Copy link
Member Author

lcian commented Sep 5, 2025

One thing to try would be whether it actually is the "fully qualified" name, because just concatenating the module to the name, does it also include the type name for impl methods?

It seems it's just the module path, which doesn't include any information about trait impls.
Seems something it could make sense to propose upstream though.

cursor[bot]

This comment was marked as outdated.

@lcian
Copy link
Member Author

lcian commented Sep 9, 2025

Found some discussions around this topic
tokio-rs/tracing#2116
tokio-rs/tracing#1939
tokio-rs/tracing#1123

@lcian
Copy link
Member Author

lcian commented Sep 10, 2025

Some still open points:

  • Do we want sentry.name or sentry.description? To clarify how we plan to move forward with this internally, also considering span first
  • We should probably attach the remaining OTEL attributes to indicate module, line number, etc. to surface the module and other metadata as discussed above. Unfortunately there seems to be no reliable way to get the fully-qualified (or even not fully qualified) name of the function. See https://opentelemetry.io/docs/specs/semconv/registry/attributes/code/ for the attribute definitions

Copy link
Member

@szokeasaurusrex szokeasaurusrex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I think this is nice, and seems like a reasonable change, as long as we don't expect it to be extremely disruptive to users when migrating over.

I added some small, optional comments, mostly with suggestions to improve documentation, tests, and code understandability. But overall, I think what you are doing makes sense!

@lcian
Copy link
Member Author

lcian commented Sep 12, 2025

Do we want sentry.name or sentry.description? To clarify how we plan to move forward with this internally, also considering span first

JS which already has a span only tracing API uses name and not description anymore, so name is fine

@lcian
Copy link
Member Author

lcian commented Sep 12, 2025

Added default attributes in 3658945 (#887)
(target, plus code attributes defined in OTEL)
Unfortunately we have to allocate for them, because span data is typed as serde_json::Value, so we need a String.

Here's how a trace looks like in Sentry: https://sentry-sdks.sentry.io/explore/traces/trace/8d4be80920b04feb03ac20aaa75f7942/?fov=0%2C697.409912109375&mode=samples&node=span-c86947b544360096&pageEnd&pageStart&project=4508694563782656&source=traces&statsPeriod=1h&table=trace&timestamp=1757679523.3

@lcian lcian enabled auto-merge (squash) September 12, 2025 12:58
@lcian lcian disabled auto-merge September 12, 2025 12:58
@lcian lcian enabled auto-merge (squash) September 12, 2025 12:58
@lcian lcian merged commit 5c8ab31 into master Sep 12, 2025
19 checks passed
@lcian lcian deleted the lcian/ref/tracing-attributes branch September 12, 2025 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tracing::span "message" field should NOT be empty during span creation Sentry span.description requires setting "message" field
3 participants